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ABSTRACT 



The Third International Mathematics and Science Study 
(TIMSS) is the largest and most ambitious international comparative study 
designed to provide policy makers, educators, researchers, and practitioners 
with information about educational achievement and learning contexts. Data 
were collected from 45 countries in more than 30 different languages. Five 
grade levels were tested in the two subject areas, totaling more than half a 
million students tested around the world. The six content dimensions covered 
in the TIMSS mathematics tests given to the primary school students include : 
whole numbers; fractions and proportionality; measurement, estimation, and 
number sense; data representation, analysis, and probability; geometry; and 
patterns, relations, and functions. The results in this report describe 
students ' mathematics achievement at both the third and fourth grade levels . 
About one quarter of the questions were in the free- response format requiring 
students to generate and write their answers. Chapters include: (1) 

"International Student Achievement in Mathematics"; (2) "Average Achievement 
in the Mathematics Content Areas"; (3) Performance on Items within Each 
Mathematics Content Area"; (4) "Students’ Backgrounds and Attitudes towards 
Mathematics"; (5) "Teachers and Instruction". Appendices contain an overview 
of TIMSS procedures, the test-curriculum matching analysis, and percentiles 
and standard deviations of mathematics achievement. ( JRH) 
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EXECUTIVE SUMMARY 



-Executive Summary 



Mathematics 



Since its inception in 1959, the International Association for the Evaluation of 
Educational Achievement (TEA) has conducted a series of international comparative 
studies designed to provide policy makers, educators, researchers, and practitioners 
with information about educational achievement and learning contexts. The Third 
International Mathematics and Science Study (TIMSS) is the largest and most 
ambitious of these studies ever undertaken. 

The scope and complexity of TIMSS is enormous. Forty-five countries collected 
data in more than 30 different languages. Five grade levels were tested in the two 
subject areas, totaling more than half a million students tested around the world. 
The success of TIMSS depended on a collaborative effort between the research 
centers in each country responsible for implementing the steps of the project and 
the network of centers responsible for managing the across-country tasks such as 
training country representatives in standardized procedures, selecting comparable 
samples of schools and students, and conducting the various steps required for data 
processing and analysis. Including the administrators in the approximately 15,000 
schools involved, many thousands of individuals around the world were involved 
in the data collection effort. Most countries collected their data in May and June 
of 1995, although those countries on a southern hemisphere schedule tested in late 
1994, which was the end of their school year. 

Six content dimensions were covered in the TIMSS mathematics tests given to the 
primary-school students: whole numbers; fractions and proportionality; measurement, 
estimation, and number sense; data representation, analysis, and probability; 
geometry; and patterns, relations, and functions. About one-fourth of the questions 
were in the free-response format requiring students to generate and write their 
answers. These types of questions, some of which required extended responses, 
were allotted approximately one-third of the testing time. Chapter 3 of this report 
contains 30 example items illustrating the range of mathematics concepts and 
processes addressed by the TIMSS test. 

Because the home, school, and national contexts within which education takes place 
can play important roles in how students learn mathematics, TIMSS collected 
extensive information about such background factors. The students who participated 
in TIMSS completed questionnaires about their home and school experiences 
related to learning mathematics. Also, teachers and school administrators completed 
questionnaires about instructional practices. System-level information was provided 
by each participating country. 
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SUMMARY 



TIMSS was conducted with attention to quality at every step of the way. Rigorous 
procedures were designed specifically to translate the tests, and numerous regional 
training sessions were held in data collection and scoring procedures. Quality control 
monitors observed testing sessions and sent reports back to the TIMSS International 
Study Center at Boston College. The samples of students selected for testing were 
scrutinized according to rigorous standards designed to prevent bias and ensure 
comparability. In this publication, the countries are grouped for reporting of achievement 
according to their compliance with the sampling guidelines and the level of their 
participation rates. Prior to analysis, the data from each country were subjected to 
exhaustive checks for adherence to the international formats as well as for within-country 
consistency and comparability across countries. 

Of the five grade levels tested, the results provided in this report describe students’ 
mathematics achievement at both the third and fourth grades. For most, but not all 
TIMSS countries, the two grades tested at the primary-school level represented 
the third and fourth years of formal schooling. Special emphasis is placed on the 
fourth-grade results, including selected information about students’ background 
experiences and teachers’ classroom practices in mathematics. Results are reported 
for the 26 countries that completed all of the steps on the schedule necessary to appear 
in this report. 

The mathematics achievement results for students in the seventh and eighth grades 
were published in Mathematics Achievement in the Middle School Years: IEA’s 
Third International Mathematics and Science Study} This report describes mathematics 
achievement in 41 countries, including results for major content areas, breakdowns 
by gender, example items, and results for selected background and attitudinal factors. 
Achievement results for students in their final year of secondary school will appear 
in a subsequent report. 

The following sections summarize the major findings described in this report. 



BEST COPY AVAILABLE 




Beaton, A.E., Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., Kelly, D.L., and Smith, T.A. (1996). Mathematics 
Achievement in the Middle School Years: lEA's Third International Mathematics and Science Study (TIMSS). 
Chestnut Hill, MA: Boston College. The science achievement results far seventh- and eighth-grade students 
are presented in a Campanian volume, Beaton, A.E., Martin, M.O., Mullis, I.V.S., Gonzalez, E.J., Smith, T.A., 
and Kelly, D.L (1996). Science Achievement in the Middle School Years: lEA's Third International Mathematics 
and Science Study (TIMSS). Chestnut Hill, MA: Boston Calleqe. 
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Students' Mathematics Achievement 

£> Singapore and Korea were the top-performing countries at both the fourth 
and third grades. Japan and Hong Kong also performed very well at both 
grades, as did the Netherlands, the Czech Republic, and Austria. Lower- 
performing countries included Iran and Kuwait (see Tables 1.1 and 1.2; 
Figures 1.1 and 1.2). 

£> Perhaps the most striking finding was the large difference in average 
achievement between the top-performing and bottom-performing 
countries. Despite this large difference, when countries were ordered by 
average achievement there were only small or negligible differences in 
achievement between one country and the one with the next-lowest 
average achievement. In some sense, at both grades, the results provide 
a chain of overlapping performances, where most countries had average 
achievement similar to a cluster of other countries, but from the beginning 
to the end of the chain there were substantial differences. For example, 
at both grades, average achievement in Singapore and Korea was 
comparable to or even exceeded performance for 95% of the students in 
the lowest-performing countries. 

£> Many countries (9 of 12) that performed above the international average 
at the fourth grade also did so at the eighth grade. However, at the eighth 
grade, Ireland and Australia were about at the international average, while 
the United States was below it (see Figure 1.3). 

£> For most countries, gender differences in mathematics achievement were 
small or essentially non-existent. However, the direction of the few 
gender differences that did exist favored boys rather than girls. Similarly, 
within the mathematics content areas, there were few differences in 
performance between boys and girls, except in measurement, where the 
differences favored boys. 

£> Compared with their overall performance in mathematics, nearly all 
countries did relatively better in several content areas than they did in 
others. This is consistent with the idea of countries having different 
curricular emphases in mathematics. 



P> Even though students in the top-performing countries had very high 
achievement on many of the test questions, students generally had the 
most difficulty with the items in the content area of fractions and 
proportionality. The least difficult items involved whole number 
proportional reasoning and recognizing pictorial representations of 
fractions. In contrast, the more difficult items involved decimals, and 
students being asked to explain their reasoning. 
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In data representation, students had some difficulty moving beyond a 
straight-forward reading of data in tables, charts, and graphs to actually 
using the information in calculations or to graphically represent the data. 
For example, students were asked to use data from a simple table to 
complete a bar graph. On average, 40% of the fourth graders and 23% of 
the third graders across countries drew the four bars to appropriate heights. 
There was, however, a very large range in performance from country to 
country. For example, about three-fourths or more of the fourth graders 
completed the bar graph in Hong Kong, Japan, Korea, and Singapore. 

Similarly, students were more likely to be able to recognize simple patterns 
and relationships than they were to determine the operations underlying 
the relationships. About half the students internationally provided an 
answer showing that they understood what to do to get the next number 
in a subtraction series, where the numbers were decreasing by 4. When 
given two columns of four numbers, only about one-fourth of the third 
graders and two-fifths of the fourth graders correctly determined that 
you needed to divide the number in Column A by 5 to obtain the number 
next to it in Column B. 



Students' Attitudes Towards Mathematics 



Those students who reported either liking mathematics or liking it a lot 
generally had higher achievement than students who reported disliking it 
to some degree. The overwhelming majority of fourth graders in nearly 
every country indicated they liked mathematics to some degree, but not 
all students feel positive about this subject area. In Japan, Korea, and the 
Netherlands, more than one-quarter of the fourth-grade students reported 
disliking mathematics. 

In most countries, fourth graders of both genders were equally positive 
about liking mathematics. In Austria, Hong Kong, Japan, and the 
Netherlands, boys reported a significantly stronger liking of the subject 
area than did girls. However, girls reported liking mathematics better 
than did boys in Ireland and Scotland. 



Across countries, the majority of fourth graders agreed or strongly agreed 
that they did well in mathematics - a perception that did not always 
coincide with the comparisons of achievement across countries on the 
TIMSS test. Fourth-grade girls had lower self-perceptions than did boys 
in Austria, Hong Kong, Japan, the Netherlands, Singapore, and Slovenia. 




15 



EXECUTIVE 



SUMMARY 



Home Environment 

Home factors were strongly related to mathematics achievement in every country 
that participated in TIMSS. 



In many countries, fourth-grade students who reported having more 
educational resources in the home had higher mathematics achievement 
than those who reported little access to such resources. Strong positive 
relationships were found between mathematics achievement and having 
study aids in the home, including a dictionary, a computer, and a study 
desk/table for the student’s own use. 

[> The number of books in the home can be an indicator of a home envi- 
ronment that values and provides general academic support. In nearly all 
countries, students reporting more than 100 books in the home had 
higher mathematics achievement than students reporting fewer books. 

[> In all but a few countries, 80% or more of the students responded that 
they always or almost always spoke the language they were tested in at 
home. Most certainly, these relatively high percentages reflect the effort 
expended by the participating countries to test in more than one lan- 
guage when necessary. 

£> In about half the countries, 80% or more of the fourth graders reported 
that both their parents were bom in that country. Yet, the patterns in 
relation to mathematics achievement varied substantially from country 
to country. The fourth graders themselves generally were bom in the 
country in which they were tested. 

£> For normal school days, fourth-grade students in most countries re- 
ported averaging approximately an hour (.7 to 1.3 of an hour) outside of 
school each day studying or doing homework in mathematics. 

P> Fourth-grade students in all countries also reported that they normally 
averaged an hour or two each school day watching television. In nearly 
all countries, students watching more than four hours of television per 
day had lower mathematics achievement than their classmates who 
watched less television. 



£> Besides watching television, students reported spending from one to two 
hours per school day playing or talking with friends and one to two 
hours per school day playing sports. (It should be noted, however, that 
the time spent in these activities is not additive because students can talk 
with their friends at sporting events or while watching TV, for example.) 
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SUMMARY 



EXECUTIVE 



Instructional Contexts and Practices 

In comparison with the positive relationships observed between mathematics 
achievement and home factors, the relationships were less clear between achievement 
and various instructional variables, both within and across countries. The interaction 
among instructional variables can be extremely complex and merits further study. 



The qualifications required for teaching certification were relatively 
uniform across countries. Most countries reported that three or four 
years of post-secondary education were required, in either a university, a 
teacher training institution, or both. Almost all countries reported that 
teaching practice was a requirement, as was an examination or evaluation. 

In most countries, the mathematics teaching force was predominantly 
female. Ninety percent or more of the fourth-grade students had female 
teachers in the Czech Republic, Hungary, Israel, Latvia (LSS), Portugal, 
Scotland, and Slovenia. 

Teachers in most countries reported that mathematics classes typically 
meet for three or four hours a week, on average. However, more than 5 
hours of weekly class time was reported for 50% or more of the fourth- 
grade students in the Netherlands, Portugal, Singapore, and Thailand. 
The data, however, revealed no clear pattern between the number of in-class 
instructional hours and mathematics achievement. 

In most countries, the challenge of catering to students of different 
academic abilities was the factor teachers mentioned most often as 
limiting how they teach their mathematics classes. Other limiting factors 
were a high student/teacher ratio, a shortage of equipment for use in 
instruction, and the burden of dealing with disruptive students. 

There was considerable variation in class size for the TIMSS countries, 
with the average ranging from 19 in Norway to 43 in Korea. In a number 
of countries, however, nearly all students (90% or more) were in classes 
of fewer than 30 students. At the other end of the spectrum, more than 
90% of the students in Korea and Singapore were in classes with more 
than 30 students. The TIMSS data showed different patterns of mathematics 
achievement in relation to class size for different countries. 



Small-group work was used less frequently than other instructional 
approaches. Across countries, mathematics teachers reported that 
working together as a class with the teacher teaching the whole class, 
and having students work individually with assistance from the teacher 
were the most frequently used instructional approaches. 




EXECUTIVE SUMMARY 



Across countries, teachers for the majority of the students reported 
being fairly familiar with the official national and/or regional curriculum 
guides in mathematics. Teachers generally reported relying on these 
guides in deciding which topics to teach. The textbook was the major 
written source mathematics teachers used in deciding how to present a 
topic to their classes. In most participating countries, teachers reported 
using a textbook in teaching mathematics for 95% or more of the 
students. 



Relatively uniformly, the majority of students were asked both to 
practice computation and to do some type of reasoning tasks in most or 
every lesson. Using things from everyday life in solving mathematics 
problems most typically is done in some lessons. 

According to teachers in many of the TIMSS countries, most fourth- 
grade students never or hardly ever use calculators in their mathematics 
classes. The exceptions where there was at least weekly use of calcula- 
tors for the majority of the students included Australia, England, and 
New Zealand. Both teachers and students agreed that the computer was 
almost never used in most students’ mathematics lessons. 



Internationally, most fourth-grade students were assigned mathematics 
homework at least once or twice a week, if not more often. Most typi- 
cally, for the majority of the students, the assignments were 30 minutes 
or less in length. In all participating countries, for at least 70% of the 
students, teachers reported at least sometimes, if not always, correcting 
homework assignments and returning those assignments to students. 

Yet, in general for the TIMSS countries, teachers reported that math- 
ematics homework assignments contributed only rarely or sometimes to 
students’ grades or marks. 
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INTRODUCTION 



Introduction 



Mathematics 



The fact that skills in mathematics and science are so critical to economic progress 
in a technologically-based society has led countries to seek information about 
what their school-age populations know and can do in mathematics and science. 
There is interest in what concepts students understand, how well they can apply 
their knowledge to problem-solving situations, and whether they can communicate 
their understanding. Even more vital, countries desire to further their knowledge 
about what can be done to improve students’ understanding of mathematical 
concepts, their ability to solve problems, and their attitudes towards learning. 

The Third International Mathematics and Science Study (TIMSS) provided countries 
with a vehicle for investigating these issues while expanding their perspectives of 
what is possible beyond the confines of their national borders. It is the most 
ambitious and complex comparative education study in a series of such undertakings 
conducted during the past 37 years by the International Association for the Evaluation 
of Educational Achievement (IEA). 1 The main purpose of TIMSS was to focus on 
educational policies, practices, and outcomes in order to enhance mathematics and 
science learning within and across systems of education. 

With its wealth of information covering more than half a million students at five 
grade levels in 15,000 schools and more than 40 countries around the world, TIMSS 
offers an unprecedented opportunity to examine similarities and differences in how 
mathematics and science education works and how well it works. The study used 
innovative testing approaches and collected extensive information about the contexts 
within which students learn mathematics and science. 

The present report focuses on the mathematics achievement of primary-school 
students in 26 countries. Participants were to test students in the two grades with 
the largest proportion of 9-year-olds - the third and fourth grades in most countries. 
Special emphasis is placed on the fourth-grade results, including selected information 
about students’ background and about classroom practices in teaching mathematics. 

The countries that participated in TIMSS tested students in both mathematics and 
science. A companion report, Science Achievement in the Primary School Years: 
IEA’ s Third International Mathematics and Science Study (TIMSS), 2 presents 
corresponding results about students’ science achievement in the lower grades. 



ERIC 



1 The previous IEA mathematics studies were conducted in 1964 and 1980-82, and the science studies in 
1970-71 and 1983-84. For information about TIMSS procedures, see Appendix A. 

2 Martin, M.O., Mullis, I.V.S., Beaton, A.E., Gonzalez, E.J., Smith, T.A., and Kelly, D.L. (1997). Science 
Achievement in the Primary School Years: lEA's Third International Mathematics and Science Study (TIMSS). 
Chestnut Hill, MA: Boston College. 
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Forty-one countries, including those in this report, also tested the mathematics and 
science achievement of students in the two grades with the largest proportion of 13- 
year-olds (seventh and eighth grades in most countries). The initial achievement 
results for the seventh- and eighth-grade students already have been published in 
two companion volumes: 3 

• Mathematics Achievement in the Middle School Years: 
lEA’s Third International Mathematics and Science Study. 

• Science Achievement in the Middle School Years: 

lEA’s Third International Mathematics and Science Study. 

Approximately 25 of the TIMSS participants also assessed the mathematics and 
science literacy of students in their final year of secondary education. Additionally, 
separate samples of students who had taken the relevant coursework were assessed 
in advanced mathematics and physics. In yet another effort, subsets of students, 
except the final-year students, also had the opportunity to participate in a “hands-on” 
performance assessment where they designed experiments and tested hypotheses. 
The achievement results for the final-year students and for the TIMSS performance 
assessment will be presented in forthcoming reports. 

Together with the achievement tests, TIMSS administered a broad array of background 
questionnaires. The data collected from students, teachers, and school principals, as 
well as the system-level information collected from the participating countries, 
provide an abundance of information for further study and research. TIMSS data 
make it possible to examine differences in current levels of performance in relation 
to a wide variety of variables associated with the classroom, school, and national 
contexts within which education takes place. 



Which Countries Participated? 

TIMSS was very much a collaborative process among countries. Table 1 shows the 
countries participating in the TIMSS testing at the primary grades. Each participant 
designated a national center to conduct the activities of the study and a National 
Research Coordinator (NRC) to assume responsibility for the successful completion 
of these tasks. 4 For the sake of comparability, all testing was conducted towards the 
end of the school year. The four countries on a Southern Hemisphere school schedule 
(Australia, Korea, New Zealand, and Singapore) tested in September through 
November of 1994, which was the end of their school year. The remaining countries 
tested the mathematics and science achievement of their students towards the end of 




3 Beaton, A.E., Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., Kelly, D.L., and Smith, T.A. (1996). Mathematics 
Achievement in the Middle School Years: lEA's Third International Mathematics and Science Study (TIMSS). 
Chestnut Hill, MA: Boston College. Beaton, A.E., Martin, M.O., Mullis, I.V.S., Gonzalez, E.J., Smith, T.A., and 
Kelly, D.l. (1996). Science Achievement in the Middle School Years: lEA's Third International Mathematics 
and Science Study (TIMSS). Chestnut Hill, MA: Boston College. 

4 Appendix D lists the National Research Coordinators as well as the members of the TIMSS advisory committees. 
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TIMSS Countries Testing in the Primary Grades 1 



Australia 


• Japan 


Austria 


• Korea 


Canada 


• Kuwait 


Cyprus 


• Latvia 


Czech Republic 


• Mexico 


England 


• Netherlands 


Greece 


• New Zealand 


Hong Kong 


• Norway 


Hungary 


• Portugal 


Iceland 


• Scotland 


Indonesia 


• Singapore 


Iran, Islamic Republic 


• Slovenia 


Ireland 


•Thailand 


Israel 


• United States 


Italy 





1 Indonesia and Italy were unable to complete the steps necessary for their data to appear in this report. 

Please see Appendix A, Figure A.1 , for countries participating in other components of the TIMSS achievement testing. 

Mexico participated in the testing portion of TIMSS, but chose not to release its results at grades 3 and 4 in the international report. 
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the 1994-95 school year, most often in May and June of 1995. Because Italy and 
Indonesia were unable to complete the steps necessary for their inclusion in this 
report, the tables throughout the report do not include data for these two countries. 
Results are also not presented for Mexico which chose not to release its third- and 
fourth-grade results in the international reports. 

Table 2 shows information about the lower and upper grades tested in each country, 
including the country names for those two grades and the years of formal schooling 
students in those grades had completed when they were tested for TIMSS. Table 2 
reveals that for most, but not all, countries, the two grades tested represented the 
third and fourth years of formal schooling. Thus, solely for convenience, the report 
often refers to the upper grade tested as the fourth grade and the lower grade tested 
as the third grade. Two countries, Israel and Kuwait, tested only at the upper grade. 

Having valid and efficient samples in each country is crucial to the quality and 
success of any international comparative study. The accuracy of the survey results 
depends on the quality of sampling information available, and particularly on the 
quality of the samples. TIMSS developed procedures and guidelines to ensure that 
the national samples were of the highest quality possible. Standards for coverage of 
the target population, participation rates, and the age of students were established, 
as were clearly documented procedures on how to obtain the national samples. For 
the most part, the national samples were drawn in accordance with the TIMSS 
standards, and achievement results can be compared with confidence. However, 
despite efforts to meet the TIMSS specifications, some countries did not do so. 
These countries are specially annotated and/or shown in separate sections of the 
tables in this report. 5 





The TIMSS sampling requirements and the outcomes af the sampling procedures are described 



in Appendix A. 
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Table 2 



Information About the Grades Tested 




2 Australia 


3 or 4 


3 or 4 


4 or 5 


4 or 5 


Austria 


3 


3 


4 


4 


Canada 


3 


3 


4 


4 


Cyprus 


3 


3 


4 


4 


Czech Republic 


3 


3 


4 


4 


England 


Year 4 


4 


Year 5 


5 


Greece 


3 


3 


4 


4 


Hong Kong 


Primary 3 


3 


Primary 4 


4 


Hungary 


3 


3 


4 


4 


Iceland 


3 


3 


4 


4 


Iran, Islamic Rep. 


3 


3 


4 


4 


Ireland 


3rd Class 


3 


4th Class 


4 


Israel 


- 


- 


4 


4 


3 Japan 


3rd Grade 


3 


4th Grade 


4 


Korea 


3rd Grade 


3 


4th Grade 


4 


Kuwait 


- 


- 


5 


5 


Latvia 


3 


3 


4 


4 


4 Netherlands 


5 


3 


6 


4 


5 New Zealand 


Standard 2 


3. 5-4.5 


Standard 3 


4.S-5.5 


Norway 


2 


2 


3 


3 


Portugal 


3 


3 


4 


4 


Scotland 


Year 4 


4 


Year 5 


5 


Singapore 


Primary 3 


3 


Primary 4 


4 


Slovenia 


3 


3 


4 


4 


Thailand 


Primary 3 


3 


Primary 4 


4 


United States 


3 


3 


4 


4 



1 Years of schooling based on the number of years children in the grade level have been in formal schooling, beginning with 
primary education (International Standard Classification of Education Level 1). Does not include preprimary education. 

2 Australia: Each state/territory has its own policy regarding age of entry to primary school. In 4 of the 8 states/territories 
students were sampled from grades 3 and 4; in the other four states/territories students were sampled from grades 4 and 5. 

3 Japan: 3rd Grade Elementary and 4th Grade Elementary 

4 In the Netherlands kindergarten is integrated with primary education. Grade-counting starts at age 4 (formerly 
kindergarten 1). Formal schooling in reading, writing, and arithmetic starts in grade 3, age 6. 

5 New Zealand: The majority of students begin primary school on or near their 5th birthday so the "years of formal schooling" vary. 

SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1994-95. Information provided by TIMSS 

National Research Coordinators. 
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What Was the Nature of the Mathematics Test? 

Together with the quality of the samples, the quality of the test also receives considerable 
scrutiny in any comparative study. All participants wish to ensure that the achievement 
items are appropriate for their students and reflect their current curriculum. Developing 
the TIMSS tests was a cooperative venture involving all of the NRCs during the entire 
process. Through a series of efforts, countries submitted items that were reviewed 
by mathematics subject-matter specialists, and additional items were written to ensure 
that the desired mathematics topics were covered adequately. Items were piloted, 
the results reviewed, and new items were written and piloted. The resulting TIMSS 
mathematics test contained 102 items representing a range of mathematics topics 
and skills. 

The TIMSS curriculum frameworks described the content dimensions for the TIMSS 
tests as well as performance expectations (behaviors that might be expected of students 
in school mathematics). 6 Six content areas are covered in the mathematics test taken 
by third- and fourth-grade students. These areas and the percentage of the test items 
devoted to each are: whole numbers (25%), fractions and proportionality (21%); 
measurement, estimation, and number sense (20%); data representation, analysis, and 
probability (12%); geometry (14%); and patterns, relations, and functions (10%). 
The performance expectations include: knowing (41%); performing routine procedures 
(16%); using complex procedures (24%); and solving problems (20%). 

About one-fourth of the questions were in the free-response format, requiring students 
to generate and write their answers. These questions, some of which required extended 
responses, were allotted approximately one-third of the testing time. Responses to 
the free-response questions were evaluated to capture diagnostic information, and 
some were scored using procedures that permitted partial credit. 7 Chapter 3 of this 
report contains 30 example items illustrating the range of mathematics concepts and 
processes addressed by the TIMSS test. 

The TIMSS tests were prepared in English and translated into the necessary additional 
languages using explicit guidelines and procedures. A series of verification checks 
were conducted to ensure the comparability of the translations. 8 




6 Robitoille, D.F., McKnight, C.C., Schmidt, W.H., Britton, E.D., Roizen, S.A., ond Nicol, C. (1993). TIMSS 
Monograph No. 1: Curriculum Frameworks for Mathematics and Science. Vancouver, B.C.: Pacific 
Educotionol Press. 

7 TIMSS scoring reliability studies within ond across countries indicate that the percent of exact agreement for 
correctness scores overoged well above 90%. For more details, see Appendix A. 

8 See Appendix A for more information about the translation procedures. 

24 



INTRODUCTION 



The tests were given so that no one student took all of the items, which would have 
required about four hours for both mathematics and science. Instead, the tests were 
assembled in eight booklets, each containing approximately one hour of material. 
Each student took only one booklet, 9 and the items were rotated through the booklets 
so that each one was answered by a representative sample of students. 

TIMSS conducted a Test-Curriculum Matching Analysis whereby countries examined 
the TIMSS test to identify items measuring topics not addressed in their curricula. 
The analysis showed that omitting such items for each country had little effect on 
the overall pattern of achievement results across all countries. 10 



How Do Country Characteristics Differ? 

International studies of student achievement provide valuable comparative information 
about student performance and instructional practices. Along with the benefits of 
international studies, though, are challenges associated with comparing achievement 
across countries, cultures, and languages. In TIMSS, extensive efforts were made to 
attend to these issues through careful planning and documentation, cooperation among 
the participating countries, standardized procedures, and rigorous attention to quality 
control throughout. 11 

Beyond the integrity of the study procedures, the results of comparative studies such 
as TIMSS also need to be considered in light of the larger contexts in which students 
are educated and the kinds of system- wide factors that might influence students’ 
opportunity to learn. A number of these factors are more fully described in National 
Contexts for Mathematics and Science Education: An Encyclopedia of the Education 
Systems Participating in TIMSS ; 12 however, some selected demographic characteristics 
of the TIMSS countries are presented in Table 3. Table 4 contains information about 
public expenditure on education. The information in these two tables shows that some 
of the TIMSS countries are densely populated and others are more rural, some are 
large and some small, and some expend considerably more resources on education 
than others. Although these factors do not necessarily determine high or low 
performance in mathematics, they do provide a context for considering the difficulty 
of the educational task from country to country. 

Describing students’ educational opportunities also includes understanding the 
knowledge and skills that students are supposed to master. To help complete the picture 
of educational practices in the TTMSS countries, mathematics and curriculum specialists 
within each country provided detailed categorizations of their curriculum guides, 




9 Primary students were given a break during the testing sessions. Four clusters of items (37 minutes total) were 
administered prior to the break and three clusters (27 minutes total) after the break. 

10 Results of the Test-Curriculum Matching Analysis are presented in Appendix B. 

11 Appendix A contains an overview af the procedures used and cites a number of references providing details 
about TIMSS methadalagy. 

12 Rabitaille, D.F. (Ed.). (1997). Notional Contexts for Mathematics and Science Education: An Encyclopedia of 
the Education Systems Participating in TIMSS. Vancouver, B.C.: Pacific Educational Press. 
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Table 3 



Selected Demographic Characteristics of TIMSS Countries 



Country 


Population 

Size 

(1,000)’ 


Area of 
Country 
(1000 
Square 
Kilometers) 2 


Density 
(Population 
per Square 
Kilometer) 3 


Percentage 

of 

Population 
Living in 
Urban 
Areas 


Life 

Expectancy 4 


Percent in 
Secondary 
School 5 


Australia 


17843 


7713 


2.29 


84.8 


77 


84 


Austria 


8028 


84 


95.28 


55.5 


77 


107 


Canada 


29248 


9976 


2.90 


76.7 


78 


88 


Cyprus 


726 


9 


77.62 


53.6 


77 


95 


Czech Republic 


10333 


79 


130.99 


65.3 


73 


86 


6 England 


48533 


130 


373.33 


- 


77 


- 


Greece 


10426 


132 


78.63 


64.7 


78 


99 


7 Hong Kong 


6061 


1 


5691 .35 


94.8 


78 


98 


Hungary 


10261 


93 


110.03 


64.2 


70 


81 


Iceland 


266 


103 


2.56 


91.4 


79 


103 


Iran 


62550 


1648 


36.98 


58.5 


68 


66 


Ireland 


3571 


70 


50.70 


57.4 


76 


105 


Israel 


5383 


21 


252.14 


90.5 


77 


87 


Japan 


124961 


378 


329.63 


77.5 


79 


96 


Korea 


44453 


99 


444.92 


79.8 


71 


93 


Kuwait 


1620 


18 


80.42 


96.8 


76 


60 


Latvia 


2547 


65 


40.09 


72.6 


68 


87 


Netherlands 


15381 


37 


409.30 


88.9 


78 


93 


New Zealand 


3493 


271 


12.78 


85.8 


76 


104 


Norway 


4337 


324 


13.31 


73.0 


78 


116 


Portugal 


9902 


92 


106.95 


35.2 


75 


81 


6 Scotland 


5132 


79 


65.15 


- 


75 


- 


Singapore 


2930 


1 


4635.48 


100.0 


75 


84 


Slovenia 


1989 


20 


97.14 


62.7 


74 


85 


Thailand 


58024 


513 


111.76 


31.9 


69 


37 


United States 


260650 


9809 


27.56 


76.0 


77 


97 



1 Estimates for 1994 based, in most cases, on a de facto definition. Refugees not permanently settled in the country of asylum 
are generally considered to be part of their country of origin. 

2 Area is the total surface area in square kilometers, comprising all land area and inland waters. 

3 Density is population per square kilometer of total surface area. 

4 Number of years a newborn infant would live if prevailing patterns of mortality at its birth were to stay the same throughout its life. 

5 Gross enrollment of all ages at the secondary level as a percentage of school-age children as defined by each country. This 

may be reported in excess of 1 00% if some pupils are younger or older than the country's standard range of secondary school age. 

6 Annual Abstract of Statistics, Office of National Statistics. 

7 Number for Secondary Enrollment is from Education Department (1995) Education Indicators for the Hong Kong Education 
System {unpublished document). 

6 Registrar General for Scotland Annual Report 1995 and Scottish Abstract of Statistics 1993. 

{ - ) A dash indicates the data were unavailable. 

SOURCE: The World Bank, Social Indicators of Development, 1996. 
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